APNIC Home

RPKI technical implementation details

The Resource Public Key Infrastructure (RPKI) system consists of a number of components interacting to provide the resource certification service. There are three high level components:

  • Certification Engines hosting RPKI Certification Authorities (CA)
  • Signing Engines optionally using Hardware Signing Modules (HSM) to secure keys, and perform RFC3779 certificate signing
  • Repositories publishing RPKI outcomes via the rsync protocol

RPKI process in brief

The certification engines run a CA, and the signed outcomes are published in a repository. The signed outcomes reflect the current state of the registry database.

Certification Engines are hosted on Linux systems, and invoke the PKI cryptography from signer engines run on discrete Linux systems, which are optionally connected to an HSM. The offline certification engine talks to one CA, the APNIC online engine talks to another. The hosted engine does not use HSM backed keys for Member signing services, and its signer engine provides soft-key management.

Certification engines maintain state in MySQL using discrete databases separate from the registry database. Access to the registry, and the command-and-control function, is managed through an intermediary system, which is hosted on kumera.

The Member-hosted engine can communicate with the APNIC engine in three ways, over three public service boundaries:

  • rsync view of the repository
  • HTTP view in the MyAPNIC Member portal
  • APNIC Certification Authority

These boundaries reinforce the separation between actions done by APNIC and actions done by Members when they make requests through the MyAPNIC portal.

APNIC operates an offline root, also known as a Trust Anchor (TA) CA that bootstraps the information in an online CA, which asserts the certificates that reflect Members’ holdings. A separate engine operates as the Member-hosted system, receiving the Member certificates and operating signings through them, as the Member requests in MyAPNIC.

RPKI implementation technologies

The RPKI systems are written in Perl, using CPAN and other libraries. The PKI functions are implemented through the use of OpenSSL, using modifications written by Rob Austein (ISC) and subsequently adopted by OpenSSL into the mainstream code release from version 1.0.0 onwards. Additionally, the OpenSSL ‘engine’ facility for merging PKCS.11 HSM into OpenSSL commands is used, to provide HSM backed keys to the OpenSSL signing service.

The communications between components of the system are provided by Apache 2.2, using REST/XML forms of query and response. The logical separation of roles is reinforced by their physical separation to distinct hosts, where this is useful.

Hosts are virtualized, permitting rapid re-deployment under the normal APNIC distribution resource planning, with the exception of HSM related systems. Each HSM represents a potential single point of failure, making the physical connectivity hard to virtualize.

Note:To prevent catastrophic key loss, the offline root CA uses keys that are physically backed up on an alternate offline CA, and a backup token. These can connect to and communicate with other HSM not related to RPKI, and so are redundantly available in a cold boot sense.

Standard system daemons provide other functions, such as the rsync repository publication. Periodic jobs run on the engines for both certification and signing through the UNIX “cron” daemon.

The offline Trust Anchor CA requires manual intervention, because its HSM interactions demand a PIN entry, in the secured machine room and as an offline system. It cannot be remote commanded through web services.

APNIC hosts the MySQL databases on virtualized, replicated hardware to provide database services. Distinct MySQL databases are used to separate the RPKI data from core Registry.

Technical Standards

APNIC has implemented RPKI according to our understanding of the following RFCs and related definitions. Not all aspects of the system are bounded by formal standards; some components may be documented by their code, or by drafts or documents outside of standards body processes, such as rsync.

HSM and PKI signing

APNIC operates HSM for its self-signed Trust Anchor, and its operating CA. These HSM are not operated in FIPS-120 mode, but follow most of the usual practices including roles-based access controls, use of tokens to place the HSM into operational mode, and backups of the HSM secured offsite.

APNIC currently uses SafeNet LunaSA for HSM systems, which provides OpenSSL Engines to implement a PKCS11 signing service. Some key management functions depend on use of the SafeNet specific HSM command and control system such as key backup, restoration, export function (where permitted), and key labelling.

Keys used in the RPKI conform to RFC6485, and are 2048-bit modulus RSA key pairs with a public exponent (e) of 65,535.

APNIC will review the HSM operations in the light of the RPKI defined CP/CPS (RFC6484).

Trust Anchors from our self-signed offline CA are published according to RFC6490. This specifies that the RSA public key modulus is stated in base64, and includes a pointer to the location of a certificate containing the TA’s RFC3779 data. The RFC6490 specified ‘TAL’ format has been configured into both the ISC rsync and the RIPE NCC validator systems, and has proven effective.

Both validation suites vary in their compliance checking, configuration, and behaviour. At least one upgrade has flagged incompatibilities in our published TAL, which was rectified. Future changes to validation software will prompt APNIC to check the TAL and decide how to manage any variances in behaviour.

Provisioning protocol (‘up-down’ protocol)

  • APNIC’s public CA provides up-down protocol, and is implemented according to RFC6492.
  • APNIC does not offer clients a publication point. Clients may request specific subject information access (SIA), as APNIC’s authority information access (AIA) is a given.
  • APNIC supports use of on-the-wire BPKI as well as the ISC defined ‘sign over my parent’ form of chain-validated PEM
  • BPKI management is handled out of band, typically through the APNIC Member portal or by arrangement.

RPKI products

  • PKI certificates published by the APNIC RPKI systems conform to RFC6487.
  • Signed Objects produced by the APNIC RPKI systems conform to RFC6488.
  • APNIC supports production of non-standard signed objects using object identifiers registered with IANA, reflecting drafts APNIC has published for general sign function.
  • Repository contents are recorded in Manifests, which conform to RFC6486.

APNIC does not at this time commit that manifests track all contents of a repository. The production cycle may cause content to be published, which does not yet appear on a manifest. Products may also be deliberately produced and placed in a repository that does not appear on a manifest. Future IETF activity may specify the manifest or other products as a ‘canonical’ catalog, and we will review this behaviour if and when this occurs.

Apart from certificates, CRL, and manifests, APNIC publishes Route Origin Attestations (ROAs) that conform to RFC6492.

APNIC does not produce ROAs in the APNIC CA. The Member hosted system reflects Member commands for specific origin-AS/prefix relationships. APNIC’s own Internet resources, used by the Secretariat in its various locations, are also managed through the Member portal. The APNIC CA actions in our role as registry and actions requested through the Member portal are separate.

Repository and publication

Public repository access is based on the rsync protocol, with the uniform resource identifier (URI) defined by RFC5781. Rsync is defined informally by its current code implementation.

APNIC publishes two repositories, one for direct products of the APNIC CA, and one for the products Members create through the hosted portal view of a different CA, which communicates with the APNIC CA through provisioning protocol. This separation of publication is a reflection of the separation of roles. APNIC does not support hierarchical nesting of daughter CA products at this time.

APNIC does not implement a publication protocol and therefore does not publish products signed by any child CA that lies outside of the Member portal hosting, even if they configure BPKI relationships through the Member portal.

Repository validation

APNIC’s repositories currently validate using ISC’s rsync and the RIPE NCC validation tools. This behaviour is tracked regularly. APNIC does not have its own validation tools, and does not currently offer repository validation or aggregation.

Services such as general signing require a form of validation that relates to the specific signed object, and the certification chain that lies above it, distinct from ROA validation aimed at routing security.